Temporal modeling is crucial for various video learning tasks. Most recent approaches employ either factorized (2D+1D) or joint (3D) spatial-temporal operations to extract temporal contexts from the input frames. While the former is more efficient in computation, the latter often obtains better performance. In this paper, we attribute this to a dilemma between the sufficiency and the efficiency of interactions among various positions in different frames. These interactions affect the extraction of task-relevant information shared among frames. To resolve this issue, we prove that frame-by-frame alignments have the potential to increase the mutual information between frame representations, thereby including more task-relevant information to boost effectiveness. Then we propose Alignment-guided Temporal Attention (ATA) to extend 1-dimensional temporal attention with parameter-free patch-level alignments between neighboring frames. It can act as a general plug-in for image backbones to conduct the action recognition task without any model-specific design. Extensive experiments on multiple benchmarks demonstrate the superiority and generality of our module.

神经体系结构搜索（NAS）是一种用于深度学习设计自动化的自动化体系结构工程方法，它是模型开发，选择，评估和性能估算的手动和错误过程的替代方法。但是，NAS的一个主要障碍是非常苛刻的计算资源需求和耗时的迭代，尤其是在数据集尺度时。在本文中，针对新兴视觉变压器（VIT），我们提出了NASHD，这是一种基于高度计算的监督学习模型，以对给定架构和配置的性能进行排名。与其他基于学习的方法不同，由于HDC体系结构的高平行处理，NASHD的速度更快。我们还评估了两个HDC编码方案：基于革兰氏阴性的NASHD的性能和效率。在来自不同范围的8个应用程序的Vimer-Ufo基准数据集上，NASHD记录可以对近100K视觉变压器模型的性能进行排名，而该模型的性能约为1分钟，同时仍可以通过复杂的模型来取得可比的结果。

本文解决了对预先训练的深神经网络进行排名并筛选最下游任务的重要问题。这是具有挑战性的，因为每个任务的基本模型排名只能通过微调目标数据集中的预训练模型来生成，该模型是蛮力且计算昂贵的。最近的高级方法提出了几个轻巧的可转移性指标来预测微调结果。但是，这些方法仅捕获静态表示，但忽略了微调动态。为此，本文提出了一个新的可传递性度量，称为\ textbf {s} elf-challenging \ textbf {f} isher \ textbf {d} is Criminant \ textbf {a} nalisy（\ textbf {\ textbf {sfda}）现有作品没有的有吸引力的好处。首先，SFDA可以将静态特征嵌入渔民空间中，并完善它们，以在类之间更好地分离性。其次，SFDA使用一种自我挑战的机制来鼓励不同的预训练模型来区分硬性示例。第三，SFDA可以轻松地为模型集合选择多个预训练的模型。 $ 33 $预培训的$ 11 $下游任务的$ 33 $预培训模型的广泛实验表明，在测量预训练模型的可传递性时，SFDA具有高效，有效和健壮。例如，与最先进的方法NLEEP相比，SFDA平均显示了59.1美元的增益，同时带来了$ 22.5 $ x的墙壁速度速度。该代码将在\ url {https://github.com/tencentarc/sfda}上提供。

面部反欺骗（FAS）在确保人脸识别系统中起着至关重要的作用。经验上，给定图像，在该图像的不同视图上具有更一致的输出的模型通常更好地执行，如图1所示。通过这种令人兴奋的观察，我们猜想令人鼓舞的特征符合不同视图的一致性可能是提升FAS模型的有希望的方法。在本文中，我们通过增强FAS中的嵌入级和预测级别一致性正规（EPCR）来彻底探讨这种方式。具体地，在嵌入级别，我们设计了密集的相似性损失，以最大化两个中间特征映射的所有位置之间以自我监督的方式;虽然在预测级别，我们优化了两个视图的预测之间的均方误差。值得注意的是，我们的EPCR没有注释，可以直接融入半监督的学习计划。考虑到不同的应用方案，我们进一步设计了五种不同的半监督协议，以衡量半监督的FAS技术。我们进行广泛的实验表明EPCR可以显着提高基准数据集上几个监督和半监控任务的性能。代码和协议即将发布。

近年来，着色吸引了越来越多的兴趣。经典的基于参考的方法通常依靠外部颜色图像来获得合理的结果。检索此类示例不可避免地需要大型图像数据库或在线搜索引擎。最近的基于深度学习的方法可以自动以低成本为图像着色。但是，总是伴随着不满意的文物和不连贯的颜色。在这项工作中，我们提出了GCP颜色化，以利用预审前的生成对抗网络（GAN）封装的丰富和多样化的颜色先验进行自动着色。具体而言，我们首先通过GAN编码器“检索”匹配的功能（类似于示例），然后将这些功能与功能调制量合并到着色过程中。得益于强大的生成颜色先验（GCP）和精致的设计，我们的GCP颜色可以通过单个前向传球产生生动的颜色。此外，通过修改GAN潜在代码获得多样化的结果非常方便。 GCP颜色还继承了可解释的gan的功能，并可以通过穿过甘恩潜在空间来实现可控制和平滑的过渡。广泛的实验和用户研究表明，GCP颜色比以前的作品具有出色的性能。代码可在https://github.com/tothebeginning/gcp-colorization上找到。

We introduce \textsc{PoliteRewrite} -- a dataset for polite language rewrite which is a novel sentence rewrite task. Compared with previous text style transfer tasks that can be mostly addressed by slight token- or phrase-level edits, polite language rewrite requires deep understanding and extensive sentence-level edits over an offensive and impolite sentence to deliver the same message euphemistically and politely, which is more challenging -- not only for NLP models but also for human annotators to rewrite with effort. To alleviate the human effort for efficient annotation, we first propose a novel annotation paradigm by a collaboration of human annotators and GPT-3.5 to annotate \textsc{PoliteRewrite}. The released dataset has 10K polite sentence rewrites annotated collaboratively by GPT-3.5 and human, which can be used as gold standard for training, validation and test; and 100K high-quality polite sentence rewrites by GPT-3.5 without human review. We wish this work (The dataset (10K+100K) will be released soon) could contribute to the research on more challenging sentence rewrite, and provoke more thought in future on resource annotation paradigm with the help of the large-scaled pretrained models.

We study a multi-factor block model for variable clustering and connect it to the regularized subspace clustering by formulating a distributionally robust version of the nodewise regression. To solve the latter problem, we derive a convex relaxation, provide guidance on selecting the size of the robust region, and hence the regularization weighting parameter, based on the data, and propose an ADMM algorithm for implementation. We validate our method in an extensive simulation study. Finally, we propose and apply a variant of our method to stock return data, obtain interpretable clusters that facilitate portfolio selection and compare its out-of-sample performance with other clustering methods in an empirical study.

Ensemble learning serves as a straightforward way to improve the performance of almost any machine learning algorithm. Existing deep ensemble methods usually naively train many different models and then aggregate their predictions. This is not optimal in our view from two aspects: i) Naively training multiple models adds much more computational burden, especially in the deep learning era; ii) Purely optimizing each base model without considering their interactions limits the diversity of ensemble and performance gains. We tackle these issues by proposing deep negative correlation classification (DNCC), in which the accuracy and diversity trade-off is systematically controlled by decomposing the loss function seamlessly into individual accuracy and the correlation between individual models and the ensemble. DNCC yields a deep classification ensemble where the individual estimator is both accurate and negatively correlated. Thanks to the optimized diversities, DNCC works well even when utilizing a shared network backbone, which significantly improves its efficiency when compared with most existing ensemble systems. Extensive experiments on multiple benchmark datasets and network structures demonstrate the superiority of the proposed method.

The input and output of most text generation tasks can be transformed to two sequences of tokens and they can be modeled using sequence-to-sequence learning modeling tools such as Transformers. These models are usually trained by maximizing the likelihood the output text sequence and assumes the input sequence and all gold preceding tokens are given during training, while during inference the model suffers from the exposure bias problem (i.e., it only has access to its previously predicted tokens rather gold tokens during beam search). In this paper, we propose MoCa ({\bf Mo}mentum {\bf Ca}libration) for text generation. MoCa is an online method that dynamically generates slowly evolving (but consistent) samples using a momentum moving average generator with beam search and MoCa learns to align its model scores of these samples with their actual qualities. Experiments on four text generation datasets (i.e., CNN/DailyMail, XSum, SAMSum and Gigaword) show MoCa consistently improves strong pre-trained transformers using vanilla fine-tuning and we achieve the state-of-the-art results on CNN/DailyMail and SAMSum datasets.

Graph neural networks (GNNs) have recently emerged as a promising learning paradigm in learning graph-structured data and have demonstrated wide success across various domains such as recommendation systems, social networks, and electronic design automation (EDA). Like other deep learning (DL) methods, GNNs are being deployed in sophisticated modern hardware systems, as well as dedicated accelerators. However, despite the popularity of GNNs and the recent efforts of bringing GNNs to hardware, the fault tolerance and resilience of GNNs has generally been overlooked. Inspired by the inherent algorithmic resilience of DL methods, this paper conducts, for the first time, a large-scale and empirical study of GNN resilience, aiming to understand the relationship between hardware faults and GNN accuracy. By developing a customized fault injection tool on top of PyTorch, we perform extensive fault injection experiments to various GNN models and application datasets. We observe that the error resilience of GNN models varies by orders of magnitude with respect to different models and application datasets. Further, we explore a low-cost error mitigation mechanism for GNN to enhance its resilience. This GNN resilience study aims to open up new directions and opportunities for future GNN accelerator design and architectural optimization.